[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function #88416

alvarezmelissa87 · 2021-01-14T22:53:28Z

Summary

Adds Maps embeddable map to anomaly explorer
Map is shown for jobs with 'lat_long' function when anomalies are filtered by swimcell selection
Map content if filtered by severity when severity control is updated
A map with the top anomaly is shown for up to 6 partition fields

Different types of jobs selected:

When maps or embeddable plugin are not enabled we show a message to the user and only show charts that can be shown:

NOTES

Custom styling for this type of layer should be supported by maps in the near future - this will allow us to match color of the points on the map to the severity colors.

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)

elasticmachine · 2021-01-14T22:53:31Z

Pinging @elastic/ml-ui (:ml)

...ck/plugins/ml/public/application/explorer/components/explorer_map_container/embedded_map.tsx

.../ml/public/application/explorer/components/explorer_map_container/explorer_map_container.tsx

peteharverson · 2021-01-15T15:16:29Z

Love the embedded map in the Anomaly Explorer!

Some thoughts:

did you try using multiple maps, as we have multiple charts for metric or population anomalies? So you'd see one map for each partition field value with one pair of actual / typical points. Not sure how performant multiple maps would be?
As you noted in the description above, will be nice when we can match the color of the anomaly dot to the severity.

walterra · 2021-01-15T16:07:56Z

This looks great! Agree with Pete's thought: What if we make this "just" another chart type of the charts when you drill down and click on anomalies (e.g. regular line chart, rare chart, population chart, map). For those charts we don't show typical values though just actuals, so maybe actual could be an optional layer but disabled by default to keep the data on display by default consistent with other charts?

qn895 · 2021-01-15T16:10:05Z

...ck/plugins/ml/public/application/explorer/components/explorer_map_container/embedded_map.tsx

+
+  return (
+    <div
+      className={'mlFieldDataCard__geoContent'}


I think we can remove this className here

Removed 👍

alvarezmelissa87 · 2021-01-25T23:49:55Z

This has been updated and is ready for another look when you get a chance. cc @walterra, @peteharverson 🙏

walterra

Great progress! Added some more comments.

walterra · 2021-01-26T11:37:59Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+// import { AnomalyRecordDoc } from '../../../../common/types/seriesConfig.chartData';
+
+interface Props {
+  seriesConfig?: any; // object;


Since all other charts code is still plain JS, agree to use the any here in this PR. I added an item to this charts meta-issue that we should migrate to TS #21163.

Updated in a9b5c32580d12c8a582b382049631e9709a7230a

walterra · 2021-01-26T11:40:13Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+  if (!embeddablePlugin) {
+    throw new Error('Embeddable start plugin not found');
+  }
+  if (!mapsPlugin) {
+    throw new Error('Maps start plugin not found');
+  }
+


If I remember correctly, for the file/data visualizer the maps plugin is defined as an optional plugin. Are there plans for a more graceful fallback in the UI if the plugin is not available?

You're correct - will update to optional as it should be and show a toast with a message that the plugin is disabled but let the anomaly explorer load normally otherwise 👍

walterra · 2021-01-26T11:42:37Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+    throw new Error('Maps start plugin not found');
+  }
+
+  const factory: any = embeddablePlugin.getEmbeddableFactory(MAP_SAVED_OBJECT_TYPE);


Looks like embeddablePlugin is fully typed, can we get rid of the any?

Removed in a9b5c32580d12c8a582b382049631e9709a7230a

walterra · 2021-01-26T11:44:41Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+  const factory: any = embeddablePlugin.getEmbeddableFactory(MAP_SAVED_OBJECT_TYPE);
+
+  const input: MapEmbeddableInput = {
+    id: uuid.v4(),


Based on other EUI examples, we use EUI's htmlIdGenerator is other places like this:

import { htmlIdGenerator } from '@elastic/eui'; const id = useMemo(() => htmlIdGenerator()(), []);

Updated in 478a75725eb997b45a1c62becae5c72bb4d9a1c2

walterra · 2021-01-26T11:45:15Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+      if (!factory) {
+        throw new Error('Map embeddable not found.');
+      }
+      const embeddableObject: any = await factory.create({


Would be great to get rid of that any.

Updated in 478a75725eb997b45a1c62becae5c72bb4d9a1c2

walterra · 2021-01-26T12:24:12Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+    >
+      <div
+        data-test-subj="xpack.ml.explorer.embeddedMapPanel"
+        className="embPanel__content"


Is this class necessary? I don't see any other reference to it in the ML plugin anywhere else.

Removed in 478a75725eb997b45a1c62becae5c72bb4d9a1c2

walterra · 2021-01-26T12:24:31Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

+        style={{
+          width: '100%',
+          height: '100%',
+          display: 'flex',
+          flex: '1 1 100%',
+          zIndex: 1,
+          minHeight: 0, // Absolute must for Firefox to scroll contents
+        }}


Can we move this to a custom class in an SCSS file?

walterra · 2021-01-26T12:33:20Z

.../plugins/ml/public/application/explorer/explorer_charts/explorer_charts_container_service.js

-    .catch((error) => {
-      console.error(error);
-    });
+  if (!isGeoMap) {


I hit an error when I try to view different types of jobs (e.g. high-count and geomap), a click in the overall swimlane results in this error:

Since this file takes care of getting data for all charts it needs to consider the possibility of different types of charts.

Should be fixed by 478a75725eb997b45a1c62becae5c72bb4d9a1c2

x-pack/plugins/ml/common/util/job_utils.ts

x-pack/plugins/ml/kibana.json

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_chart_embedded_map.tsx

peteharverson · 2021-01-26T17:20:37Z

Looks like there is an issue with display of the detector function in the chart info tooltip for lat_long detectors. It says null where it should be lat_long:

Thanks for catching that, @peteharverson - fixed in 478a75725eb997b45a1c62becae5c72bb4d9a1c2

x-pack/plugins/ml/common/util/job_utils.ts

x-pack/plugins/ml/public/application/explorer/explorer_charts/map_config.ts

x-pack/plugins/ml/common/util/job_utils.ts

peteharverson · 2021-01-28T10:33:32Z

x-pack/plugins/ml/public/application/explorer/explorer_charts/explorer_charts_container.js

-            )}
-          </MlTooltipComponent>
-        );
+        if (chartType === CHART_TYPE.SINGLE_METRIC) {


Probably one for a follow up, but would be nice if the SINGLE_METRIC and DISTRIBUTION charts expanded to the height of the map in this situation (preferable than shrinking the map height to match those chart heights I think).

qn895 · 2021-01-28T17:37:07Z

LGTM 🎉

kibanamachine · 2021-01-28T18:46:20Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 4fed952

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`ml`	1728	1730	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`ml`	6.5MB	6.5MB	+5.7KB

History

💚 Build #101644 succeeded c7b782b
💚 Build #101633 succeeded 1912759
💚 Build #101571 succeeded ea9414a
💚 Build #101560 succeeded 3737edb5f45df05517675075674383d901b4f7e2
💔 Build #101338 failed 478a75725eb997b45a1c62becae5c72bb4d9a1c2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

peteharverson

Tested latest changes, and LGTM

…lat_long' function (#88416) (#89738) * wip: create embedded map component for explorer * add embeddedMap component to explorer * use geo_results * remove charts callout when map is shown * add translation, round geo coordinates * create GEO_MAP chart type and move embedded map to charts area * remove embedded map that is no longer used * fix type and fail silently if plugin not available * fix multiple type of jobs charts view * fix tooltip function and remove single viewer link for latlong * ensure diff types of jobs show correct charts. fix jest test * show errorCallout if maps not enabled and is lat_long job * use shared MlEmbeddedMapComponent in explorer * ensure latLong jobs not viewable in single metric viewer * update jest test

alvarezmelissa87 added release_note:enhancement :ml Feature:Anomaly Detection ML anomaly detection v8.0.0 v7.12.0 labels Jan 14, 2021

alvarezmelissa87 self-assigned this Jan 14, 2021

alvarezmelissa87 requested a review from a team as a code owner January 14, 2021 22:53

alvarezmelissa87 requested review from thomasneirynck, peteharverson, walterra and qn895 January 15, 2021 02:11

peteharverson reviewed Jan 15, 2021

View reviewed changes

...ck/plugins/ml/public/application/explorer/components/explorer_map_container/embedded_map.tsx Outdated Show resolved Hide resolved

.../ml/public/application/explorer/components/explorer_map_container/explorer_map_container.tsx Outdated Show resolved Hide resolved

qn895 reviewed Jan 15, 2021

View reviewed changes

alvarezmelissa87 changed the title ~~[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function~~ WIP[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function Jan 15, 2021

alvarezmelissa87 force-pushed the ml-add-explorer-maps branch from 398e64e to 90141a2 Compare January 25, 2021 17:23

alvarezmelissa87 changed the title ~~WIP[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function~~ [ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function Jan 25, 2021

thomasneirynck mentioned this pull request Jan 26, 2021

[Maps] Implement fields and bounds retrieval on GeoJsonFileSource #88294

Merged

1 task

walterra mentioned this pull request Jan 26, 2021

[ML] Add visuals in result dashboards for all ML detector functions #22922

Open

61 tasks

walterra reviewed Jan 26, 2021

View reviewed changes

peteharverson reviewed Jan 26, 2021

View reviewed changes

x-pack/plugins/ml/common/util/job_utils.ts Outdated Show resolved Hide resolved

alvarezmelissa87 force-pushed the ml-add-explorer-maps branch from 33a7908 to dff81a9 Compare January 26, 2021 19:45

alvarezmelissa87 added 3 commits January 27, 2021 13:25

wip: create embedded map component for explorer

cc12e5e

add embeddedMap component to explorer

0e9b281

use geo_results

052014c

alvarezmelissa87 added 8 commits January 27, 2021 13:25

remove charts callout when map is shown

8d0cf05

add translation, round geo coordinates

d2ecf50

create GEO_MAP chart type and move embedded map to charts area

ebce0ac

remove embedded map that is no longer used

ffb319f

fix type and fail silently if plugin not available

d6232f6

fix multiple type of jobs charts view

423e12d

fix tooltip function and remove single viewer link for latlong

bff94a7

ensure diff types of jobs show correct charts. fix jest test

ea9414a

alvarezmelissa87 force-pushed the ml-add-explorer-maps branch from 3737edb to ea9414a Compare January 27, 2021 20:28

alvarezmelissa87 added 2 commits January 27, 2021 16:15

show errorCallout if maps not enabled and is lat_long job

1912759

use shared MlEmbeddedMapComponent in explorer

c7b782b

thomasneirynck mentioned this pull request Jan 28, 2021

[Maps] Add table source for choropleth mapping #89263

Merged

peteharverson reviewed Jan 28, 2021

View reviewed changes

x-pack/plugins/ml/public/application/explorer/explorer_charts/map_config.ts Show resolved Hide resolved

x-pack/plugins/ml/common/util/job_utils.ts Outdated Show resolved Hide resolved

peteharverson reviewed Jan 28, 2021

View reviewed changes

alvarezmelissa87 added 2 commits January 28, 2021 09:54

ensure latLong jobs not viewable in single metric viewer

4b8124d

update jest test

4fed952

qn895 approved these changes Jan 28, 2021

View reviewed changes

peteharverson approved these changes Jan 29, 2021

View reviewed changes

alvarezmelissa87 merged commit a08895d into elastic:master Jan 29, 2021

alvarezmelissa87 mentioned this pull request Jan 29, 2021

[7.x] [ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function (#88416) #89738

Merged

alvarezmelissa87 deleted the ml-add-explorer-maps branch January 29, 2021 15:44

lcawl added the release_note:feature Makes this part of the condensed release notes label Feb 5, 2021

thomasneirynck mentioned this pull request Feb 8, 2021

[ML][Maps] Add show-in-maps button to ML UX #69064

Closed

lcawl mentioned this pull request Mar 31, 2021

[DOCS] Add anomaly detection example for geographic data elastic/stack-docs#1631

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function #88416

[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function #88416

alvarezmelissa87 commented Jan 14, 2021 •

edited

Loading

elasticmachine commented Jan 14, 2021

peteharverson commented Jan 15, 2021

walterra commented Jan 15, 2021

qn895 Jan 15, 2021

alvarezmelissa87 Jan 25, 2021

alvarezmelissa87 commented Jan 25, 2021

walterra left a comment

walterra Jan 26, 2021

alvarezmelissa87 Jan 27, 2021

walterra Jan 26, 2021

alvarezmelissa87 Jan 26, 2021 •

edited

Loading

walterra Jan 26, 2021

alvarezmelissa87 Jan 27, 2021

walterra Jan 26, 2021

alvarezmelissa87 Jan 27, 2021

walterra Jan 26, 2021

alvarezmelissa87 Jan 27, 2021

walterra Jan 26, 2021

alvarezmelissa87 Jan 27, 2021 •

edited

Loading

walterra Jan 26, 2021

walterra Jan 26, 2021

alvarezmelissa87 Jan 27, 2021

peteharverson commented Jan 26, 2021 •

edited by alvarezmelissa87

Loading

peteharverson Jan 28, 2021

qn895 commented Jan 28, 2021

kibanamachine commented Jan 28, 2021

peteharverson left a comment

[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function #88416

[ML] Anomaly Detection: add anomalies map to explorer for jobs with 'lat_long' function #88416

Conversation

alvarezmelissa87 commented Jan 14, 2021 • edited Loading

Summary

NOTES

Checklist

elasticmachine commented Jan 14, 2021

peteharverson commented Jan 15, 2021

walterra commented Jan 15, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 commented Jan 25, 2021

walterra left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 Jan 26, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvarezmelissa87 Jan 27, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peteharverson commented Jan 26, 2021 • edited by alvarezmelissa87 Loading

Choose a reason for hiding this comment

qn895 commented Jan 28, 2021

kibanamachine commented Jan 28, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

History

peteharverson left a comment

Choose a reason for hiding this comment

alvarezmelissa87 commented Jan 14, 2021 •

edited

Loading

alvarezmelissa87 Jan 26, 2021 •

edited

Loading

alvarezmelissa87 Jan 27, 2021 •

edited

Loading

peteharverson commented Jan 26, 2021 •

edited by alvarezmelissa87

Loading